Palettized image compression

ABSTRACT

An adaptive entropy coder is coupled with a localized conditioning context to provide efficient compression of images with localized high frequency variations. In one implementation, an arithmetic coder can be used as the adaptive entropy coder. The localized conditioning context includes a basic context region with multiple context pixels that are adjacent the current pixel, each of the context pixels having an image tone. A state is determined for the basic context region based upon a pattern of unique image tones among the context pixels therein. An extended context region that includes the basic context region is used to identify a non-local trend within the context pixels and a corresponding state. A current pixel may be arithmetically encoded according to a previously encoded pixel having the same tone or as a not-in-context element. In one implementation, a not-in-context element may be represented by a tone in a color cache that is arranged as an ordered list of most recent not-in-context values.

RELATED APPLICATIONS

[0001] This application is a continuation of and claims priority to U.S.patent application Ser. No. 09/577,544, filed May 24, 2000, thedisclosure of which is incorporated by reference herein.

FIELD OF THE INVENTION

[0002] The present invention relates to data compression and, inparticular, to compression of digital image data.

BACKGROUND AND SUMMARY OF THE INVENTION

[0003] A wide range of methods and algorithms are available forcompression of digital images and, particularly, the image datacorresponding to the pixels in a digital image. The different availablemethods and algorithms are typically suited to particular types ofdigital images. An example of such particularized suitability is theJPEG compression standard, which is a widely-adopted standardcompression algorithm that is based upon the discrete cosine transform(DCT). JPEG compression and the DCT are described in, for example,Introduction to Data Compression by Khalid Sayood, Morgan KaufmannPublishers, Inc., San Francisco, Calif., 336-348, 1996.

[0004] JPEG compression applies the DCT to image data corresponding tosuccessive 8×8 blocks of pixels. The DCT is effective and efficient withregard to linear correlations that arise over each 8×8 block of pixels.Digital images with these characteristics might include images thatcorrespond to photographs or images with relatively large regions ofsimilar color or tone. Images corresponding to photographs typicallyinclude a substantially continuous range of image tones represented bybinary values of 24-bits or more. In many such continuous-tone digitalimages, the variations within the range of image tones are typicallywell accommodated by the DCT in JPEG compression. Images with relativelylarge regions of the same color or tone have no changes in tone overthose regions and may be effectively compressed by JPEG compression, aswell as other compression methods.

[0005] It will be appreciated, however, that continuous-tone images arenot the only types of digital images. In some applications, digitalimages may include a smaller range of discrete image colors or tones(e.g., represented by 8-bits, or fewer) or localized high frequencyvariations such as text elements or other image discontinuities. Thesetypes of discrete-tone images, sometimes called palettized images, arisein a variety of applications including, for example, training orhelp-screen images for software that uses widowed or other graphicaluser interfaces, shared whiteboards or application sharing, onlinetraining, simple animations, etc. The images that arise in suchapplications are often inefficiently encoded by JPEG compression due tothe spatially localized high frequency and discontinuous features in theimages.

[0006] The present invention includes an adaptive entropy coder coupledwith a localized conditioning context to provide efficient compressionof discrete-tone images with localized high frequency variations. In oneimplementation, an arithmetic coder can be used as the adaptive entropycoder.

[0007] The localized conditioning context includes a basic contextregion with multiple context pixels that are adjacent the current pixel,each of the context pixels having an image tone. A state is determinedfor the basic context region based upon the pattern of unique imagetones among the context pixels therein. An extended context region thatincludes the basic context region is used to identify a non-local trendwithin the context pixels and a corresponding state. The current pixelis then encoded as having one of the tones in the context or as anot-in-context element. In one implementation, a not-in-context elementmay be represented by a tone in a color cache that is arranged as anordered list of most recent not-in-context values.

[0008] Statistics for conditional probabilities are gathered for eachcontext during encoding. Many traditional context-based coders choosecontexts that are based on the values present in the surroundingcontext. Since the number of states in the coder grows exponentiallywith the size of the context, traditional context-based coders work wellonly when the possible set of values is small (e.g. black and white orbinary images), or when the context is very small (one element).

[0009] The present invention allows the use of a large context (e.g., upto at least six elements) with elements that can take on many values.This is performed by quantizing the huge number of fundamental states(about 300 trillion) into a set of meaningful patterns (e.g., 60) thatare based on the number of unique image tones or values. These patternsmodel palettized images well. Quantizing the states into this set ofmeaningful patterns, rather than scalar quantizing the values of theelements, as is traditionally done, greatly increases the efficiencywith which palettized images can be encoded or compressed.

[0010] Additional objects and advantages of the present invention willbe apparent from the detailed description of the preferred embodimentthereof, which proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0011]FIG. 1 is a block diagram of a computer system that may be used toimplement the present invention.

[0012]FIG. 2 is an illustration of a first exemplary palettized image.

[0013]FIG. 3 is an illustration of a second exemplary palettized image.

[0014]FIG. 4 is a flow diagram of a palettized image compression methodfor compressing or encoding data representing palettized images.

[0015]FIG. 5 is diagram illustrating a causal context region that is inthe vicinity of a current pixel being encoded.

[0016]FIG. 6 is a diagram of a predefined encoding pattern or sequencefor encoding a palettized image.

[0017]FIG. 7 is a flow diagram of a method of determining a state forcausal context region in the vicinity of a current pixel.

[0018]FIG. 8 illustrates a universe of image tone patternclassifications for a basic context region according to the presentinvention.

[0019]FIG. 9 illustrates four possible trend tone patterns of a pair ofsupplemental context pixels.

[0020]FIG. 10 is a flow diagram of an entropy coding method employed inone implementation of the present invention.

[0021]FIGS. 11 and 12 are diagrams illustrating operation of a colorcache employed in the arithmetic coding method of FIG. 10.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0022]FIG. 1 illustrates an operating environment for an embodiment ofthe present invention as a computer system 20 with a computer 22 thatcomprises at least one high speed processing unit (CPU) 24 inconjunction with a memory system 26, an input device 28, and an outputdevice 30. These elements are interconnected by at least one busstructure 32.

[0023] The illustrated CPU 24 is of familiar design and includes an ALU34 for performing computations, a collection of registers 36 fortemporary storage of data and instructions, and a control unit 38 forcontrolling operation of the system 20. The CPU 24 may be a processorhaving any of a variety of architectures including Alpha from Digital,MIPS from MIPS Technology, NEC, IDT, Siemens, and others, x86 from Inteland others, including Cyrix, AMD, and Nexgen, and the PowerPC from IBMand Motorola.

[0024] The memory system 26 generally includes high-speed main memory 40in the form of a medium such as random access memory (RAM) and read onlymemory (ROM) semiconductor devices, and secondary storage 42 in the formof long term storage mediums such as floppy disks, hard disks, tape,CD-ROM, flash memory, etc. and other devices that store data usingelectrical, magnetic, optical or other recording media. The main memory40 also can include video display memory for displaying images through adisplay device. Those skilled in the art will recognize that the memory26 can comprise a variety of alternative components having a variety ofstorage capacities.

[0025] The input and output devices 28 and 30 also are familiar. Theinput device 28 can comprise a keyboard, a mouse, a physical transducer(e.g., a microphone), etc. The output device 30 can comprise a display,a printer, a transducer (e.g., a speaker), etc. Some devices, such as anetwork interface or a modem, can be used as input and/or outputdevices.

[0026] As is familiar to those skilled in the art, the computer system20 further includes an operating system and at least one applicationprogram. The operating system is the set of software which controls thecomputer system's operation and the allocation of resources. Theapplication program is the set of software that performs a task desiredby the user, using computer resources made available through theoperating system. Both are resident in the illustrated memory system 26.

[0027] In accordance with the practices of persons skilled in the art ofcomputer programming, the present invention is described below withreference to acts and symbolic representations of operations that areperformed by computer system 20, unless indicated otherwise. Such actsand operations are sometimes referred to as being computer-executed andmay be associated with the operating system or the application programas appropriate. It will be appreciated that the acts and symbolicallyrepresented operations include the manipulation by the CPU 24 ofelectrical signals representing data bits which causes a resultingtransformation or reduction of the electrical signal representation, andthe maintenance of data bits at memory locations in memory system 26 tothereby reconfigure or otherwise alter the computer system's operation,as well as other processing of signals. The memory locations where databits are maintained are physical locations that have particularelectrical, magnetic, or optical properties corresponding to the databits.

[0028]FIGS. 2 and 3 are simplified diagrams illustrating palettizedimages 46 and 48, which are sometimes called discrete-tone images. Oneway to characterize a palettized image is that the number of colors inthe image is small compared to the number of pixels in the image (i.e.,the image size). Another way to characterize a palettized image is thatthe number of colors in the image is represented by N-number or fewerbinary bits, with N having a value of about 8. Generally, a palettizedimage is contrasted with a continuous tone image, such as a photograph,that potentially includes large numbers of colors or tones in comparisonto the number of pixels in the image. As an example, palettized imagesare commonly used in various computer user interfaces, including thosefor desktop software applications, Internet (e.g., World Wide Web)network sites, etc. It will be appreciated, however, that palettizedimages can arise in various other computer applications.

[0029]FIG. 4 is a flow diagram of a palettized image compression method50 for compressing or encoding image data representing palettizedimages. Palettized image compression method 50 compresses palettizedimages with high efficiency and no loss of information, and so may bereferred to as a lossless compression method. Palettized imagecompression method 50 compresses palettized images on a pixel-by-pixelbasis.

[0030] Process block 52 indicates that a “current” pixel 54 (FIG. 5) isdesignated for encoding. Successive adjacent pixels of a palettizedimage are encoded in a predefined encoding pattern or sequence 56 (FIG.6). In the illustrated implementation, the pixels of a palettized imageare encoded left-to-right across successive horizontal rows of pixels,beginning with a start pixel 58 at the upper left corner of the image.It will be appreciated that other encoding patterns or sequences couldalternatively be applied.

[0031] Process block 60 indicates that a state is determined for acausal context region 62 (FIG. 5), sometimes referred to as extendedcausal context region 62, that includes a predefined set of previouslyencoded pixels that are in the vicinity of current pixel 54, asdescribed below in greater detail. In accordance with the presentinvention, quantization of causal context region 62 into patternsgreatly reduces the number of possible states in comparison to priorencoding processes, thereby supporting the high efficiency of palettizedimage compression method 50.

[0032] Process block 64 indicates that current pixel 54 is adaptivelyentropy coded with regard to the predefined set of previously encodedpixels within causal context region 62. In one implementation, theadaptive entropy coding includes arithmetic coding, which is known inthe art and described in Introduction to Data Compression by KhalidSayood, Morgan Kaufmann Publishers, Inc., San Francisco, Calif., 1996.Other suitable adaptive entropy coding methods include adaptive Huffmancodes.

[0033]FIG. 7 is a flow diagram of a state determination method 70 fordetermining a state for causal context region 62 in the vicinity ofcurrent pixel 54.

[0034] Process block 72 indicates that a basic causal context region 74(FIG. 5) is defined with regard to current pixel 54 to be encoded. Basiccausal context region 74 corresponds to context pixels 76-1, 76-2, 76-3,and 76-4 in the palettized image that have been previously encoded andare immediately adjacent (i.e., contact) current pixel 54. Theillustrated implementation corresponds to an image in which pixels arearranged in a conventional rectangular array, which results in basiccontext region 74 having only four context pixels 76 that areimmediately adjacent current pixel 54. Also, the arrangement of contextpixels 76 relates to the illustrated left-to-right, top-to-bottomencoding pattern 56. It will be appreciated that a different pixelarrangement and a different encoding pattern could result in a differentbasic context region.

[0035] Each pixel has associated with it a value, called an image value,which corresponds to the color or tone, or other image characteristicsof the pixel. Process block 80 indicates that the image values ofcontext pixels 76 are compared to determine the pattern of unique imagevalues among the four context pixels, thereby to classify basic contextregion 74 by the pattern of unique image values or tones among the fourpixels. For example, basic context region 74 is referred to as one-tonecontext if all four context pixels have the same image value. If thereare two unique values, basic context region 74 is called a two-tonecontext, and so on. With only four context pixels 76, the image valueswithin basic context region 74 can correspond to one-, two, three, orfour-tone contexts.

[0036] Process block 82 indicates that a tone pattern of basic contextregion 74 is assigned a state according to the arrangement of uniqueimage values or tones among the four context pixels 76. FIG. 8illustrates the universe of fifteen tone patterns classifications forbasic context region 74.

[0037] In particular, a one-tone pattern has only one possibleclassification 90 that is designated as “tone 1, pattern 0”corresponding to all context pixels 76 having the same image value ortone (e.g., designated “a”). A two-tone pattern may be any of sevenclassifications 92-104 that are designated as patterns 0-6,respectively. A three-tone pattern may be any of six classifications106-116 as patterns 0-5, respectively. A four-tone pattern has only onepossible classification 118 that is designated as “tone 4, pattern 0.”

[0038] This classification of basic context region 74 by the numbertones and their arrangement is based upon context pixels 76 having equalimage values, or not. Other than this equality or inequality of theimage values or tones, the classification of basic context region 74 isindependent of the actual image values. The pattern classification isonly concerned with spatial distribution of the pattern, and not thevalues themselves. In contrast, conventional encoding methods baseregional contexts on the actual image values, not the pattern of uniquevalues. Even with a relatively simple color spectrum of 256 colors(i.e., 8-bit colors), a conventional regional context of four pixelswould encompass many millions of possible patterns, rather than the 15patterns shown in FIG. 8. As a result, classification of basic contextregion 74 according to the arrangements of unique image values or tones,rather than actual tones, provides a greatly reduced set of possiblecontext patterns, thereby providing a significant increase in encodingefficiency.

[0039] Process block 120 indicates that supplemental context pixels 76-5and 76-6 (FIG. 5) are appended to basic causal context region 74 tocomplete the extended context region 62 and to detect any non-localtrend extending through region 62. Supplemental context pixels 76-5 and76-6 are horizontally- and vertically-aligned with current pixel 54 anddetect horizontal and vertical trends, respectively. As shown in FIG. 2,for example, many palettized images may include image features withhorizontal or vertical boundaries that would correspond to non-localtrends through overall context region 62. A horizontal trend isindicated when context pixel 76-4 has an image value or tone equalingthat of context pixel 76-5. A vertical trend is indicated when contextpixel 76-3 has an image value or tone equaling that of context pixel76-6.

[0040]FIG. 9 illustrates the four possible trend tone patterns 124-130of supplemental context pixels 76-5 and 76-6. In trend tone pattern 124,neither of supplemental context pixels 76-5 and 76-6 has the same toneas either of respective context pixels 764 and 76-2, therebycorresponding to no trends. In trend tone pattern 126, supplementalcontext pixel 76-5 has the same tone context pixel 76-4, but pixels 76-6and 76-2 differ from each other, thereby corresponding only to ahorizontal trend. In trend tone pattern 128, supplemental context pixel76-6 has the same tone context pixel 76-2, but pixels 76-5 and 76-4differ from each other, thereby corresponding only to a vertical trend.In trend tone pattern 130, both of supplemental context pixels 76-5 and76-6 have the same tones as respective context pixels 76-4 and 76-2,thereby corresponding to both horizontal and vertical trends.

[0041] The presence or absence of horizontal or vertical trendsclassifies each pattern into another four trend states 124-130. Incombination the fifteen possible states or patterns of basic contextregion 74, the six context pixels 76-1 through 76-6 can represent atotal of sixty states into which the overall context region can beclassified.

[0042] Process block 132 indicates that a trend state is assigned tocontext region 62.

[0043]FIG. 10 is a flow diagram of an entropy coding method 150 employedin one implementation of the present invention.

[0044] Process block 152 indicates that method 150 is initialized that afirst current pixel 54 is designated.

[0045] Process block 154 indicates that a context region 62 isclassified into a selected state by, for example, state determinationmethod 70.

[0046] Process block 156 indicates that each of the unique values of thepixels or elements in the context region 62 is indexed, as known in theart of entropy coding.

[0047] Inquiry block 158 represents an inquiry as to whether currentelement 54 has a value equal to that of a previously indexed pixel orelement of the selected state. Inquiry block 158 proceeds to processblock 160 whenever current element 54 has a value equal to that of apreviously indexed pixel or element of the selected state. Inquiry block158 proceeds to process block 162 whenever current element 54 has avalue that is not equal to that of a previously indexed pixel or elementof the selected state.

[0048] Process block 160 indicates that an entropy code symbolcorresponding to the value of the previously indexed element is written,stored, or otherwise sent.

[0049] Process block 164 indicates that an estimated probability massfunction for each of the indices in the selected state is updated toreflect the symbol written in process block 160. As is known in the artof entropy coding, a probability mass function is maintained for eachsymbol being encoded. This allows the entropy coding to be adaptivelytrained to the statistics of each source, and sends shorter symbols forthe tone indices that are popular for each state. This adaptation allowsthe coder to improve its efficiency by using knowledge of previouspatterns in the image to code future similar patterns. Some examplesinclude line or checkerboard patterns that are quick to learn. Othermore complicated phenomena may include a particular font for text.

[0050] Process block 162 indicates that an entropy code symbol is sentor designated indicating a not-in-context element.

[0051] Inquiry block 170 represents an inquiry as to whether currentelement 54 has a value equal to one maintained in a color cache 172(FIGS. 11 and 12). Color cache 172 maintains an ordered list of the mostrecently used values that were out of context. The ordered list of colorcache 172 is different from a conventional list of the most commonlyused values, which does not work as well due to poor local adaptation.Inquiry block 170 proceeds to process block 174 whenever current element54 has a value equal to one maintained in color cache 172. Inquiry block170 proceeds to process block 176 whenever current element 54 does nothave a value equal to one maintained in color cache 172.

[0052] Process block 174 indicates that the index for the cache entry isdetermined, and the symbol for the index is written or stored. The indexis determined by sequentially searching the cache from most recentlyused values to least recently used values, ignoring any values that mayhave already appeared in the tones for the context.

[0053] It will be appreciated that the logical state of cache 172 is asubset of elements in the physical cache. Since the value being codedwas out of context, it by definition cannot have the same value as anyof the tones in the context. Therefore, any elements in cache 172 thathave the same value as any of the tones are not useful, and are removedfrom the logical state of cache 172. The remaining values are indexedand the probability mass function for the indices are estimated andadapted every time a value is coded out of context.

[0054] Process block 178 indicates that color cache 172 and itsestimated probability mass function are updated to reflect the symbolwritten in process block 172. As illustrated in FIG. 11, this updatingof color cache 172 includes rotating to the top the value of the symbolbeing written.

[0055] Process block 176 indicates that an entropy code symbol is sentor designated indicating a not-in-cache element.

[0056] Process block 180 indicates that the index for a color histogramentry is determined, and the symbol for the index is written or stored.The color histogram contains all the possible values for an element andkeeps track of the probabilities of all the out of cache values. Thisdistribution is used to write the symbol for the out of cache value.

[0057] Process block 182 indicates that the color histogram and itsestimated probability mass function are updated to reflect the symbolwritten in process block 180. In addition, the symbol value is placed(pushed) at the top of color cache 172, and the least recently usedvalue drops (pops) out of the bottom of cache 172 as in a queue (FIG.12).

[0058] Inquiry block 184 represents an inquiry as to whether there isanother element or pixel to be coded. Inquiry block 184 proceeds toprocess block 186 whenever there is another element or pixel to becoded. Inquiry block 184 proceeds to termination block 188 wheneverthere is not another element or pixel to be coded.

[0059] Process block 186 indicates that the method advances to the nextpixel or element. Process block 186 returns to process block 154.

[0060] Having described and illustrated the principles of our inventionwith reference to an illustrated embodiment, it will be recognized thatthe illustrated embodiment can be modified in arrangement and detailwithout departing from such principles. It should be understood that theprograms, processes, or methods described herein are not related orlimited to any particular type of computer apparatus, unless indicatedotherwise. Various types of general purpose or specialized computerapparatus may be used with or perform operations in accordance with theteachings described herein. Elements of the illustrated embodiment shownin software may be implemented in hardware and vice versa.

[0061] In view of the many possible embodiments to which the principlesof our invention may be applied, it should be recognized that thedetailed embodiments are illustrative only and should not be taken aslimiting the scope of our invention. Rather, we claim as our inventionall such embodiments as may come within the scope and spirit of thefollowing claims and equivalents thereto.

1. In one or more computer readable media, an image encoding contextregion data structure used in compressing image data representing aselected pixel out of plural pixels of plural nonzero image tones,comprising: a basic state context region data structure representingonly previously encoded pixels that are immediately adjacent theselected pixel.
 2. The media of claim 1 in which the basic state contextregion data structure represents a pattern of image tones in thepreviously encoded pixels that are immediately adjacent the selectedpixel. 3 The media of claim 1 further comprising a trend state contextregion data structure representing first and second previously encodedpixels that are immediately adjacent pixels represented by the basicstate context region data structure.
 4. The media of claim 3 in whichthe trend state context region data structure represents only first andsecond previously encoded pixels that are immediately adjacent pixelsrepresented by the basic state context region data structure.
 5. Themedia of claim 3 in which the trend state context region data structurerepresents horizontal and vertical trends with respect to the selectedpixel.
 6. A method comprising: generating an image encoding contextregion data structure used in compressing image data representing aselected pixel out of plural pixels of plural nonzero image tones,wherein the data structure generating a basic state context region datastructure representing only previously encoded pixels that areimmediately adjacent the selected pixel.
 7. The method of claim 1 inwhich the basic state context region data structure represents a patternof image tones in the previously encoded pixels that are immediatelyadjacent the selected pixel. 8 The method of claim 1 further comprisinggenerating a trend state context region data structure representingfirst and second previously encoded pixels that are immediately adjacentpixels represented by the basic state context region data structure. 9.The method of claim 8 in which the trend state context region datastructure represents only first and second previously encoded pixelsthat are immediately adjacent pixels represented by the basic statecontext region data structure.
 10. The method of claim 8 in which thetrend state context region data structure represents horizontal andvertical trends with respect to the selected pixel.